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Abstract During the conceptual and prototype design stage of an industrial 
product, it is crucial to take assembly /disassembly and maintenance opera- 
tions in advance. A well-designed system should enable relatively easy access 
of operating manipulators in the constrained environment and reduce mus- 
culoskeletal disorder risks for those manual handling operations. Trajectory 
planning comes up as an important issue for those assembly and maintenance 
operations under a constrained environment, since it determines the accessi- 
bility and the other ergonomics issues, such as muscle effort and its related 
j*V| fatigue. In this paper, a customer-oriented interactive approach is proposed to 

partially solve ergonomic related issues encountered during the design stage 
under a constrained system for the operator's convenience. Based on a sin- 
gle objective optimization method, trajectory planning for different operators 
could be generated automatically. Meanwhile, a motion capture based method 
assists the operator to guide the trajectory planning interactively when either 
0^ ' a local minimum is encountered within the single objective optimization or the 

operator prefers guiding the virtual human manually. Besides that, a physical 
engine is integrated into this approach to provide physically realistic simulation 
in real time manner, so that collision free path and related dynamic informa- 
tion could be computed to determine further muscle fatigue and accessibility 
| of a product design. 

Keywords virtual human • constrained environment design • optimization • 
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1 Introduction 



For industrial products, a compact design decreases the required massive space 
and enhances the appearance of the product. From another aspect, the de- 
signers have to consider constrained environments resulting from the compact 
design. Under constrained situations, assembly /disassembly oriented design 
has to be taken into consideration, since there are several ergonomic issues 
for the end user of the product or for the maintenance process. For example, 
the visibility [5] and accessibility [51IT2] of a component during an assembly 
operation; awkward posture caused by the product layout; physical or mental 
fatigue from the operations, etc. 

For these reasons, virtual human simulations are often engaged during the 
conceptual design stage to evaluate the accessibility of the virtual environment 
and other ergonomic aspects [4llllj. Thanks to the interaction between the 
virtual human and the constrained environment, the designers are able to 
evaluate the manual handling operations, plan the possible trajectories, and 
further improve the design. 

Trajectory planning [2] is one of the most important problems for the use 
of virtual human in product design. In general, three approaches have been 
used frequently in the literature to generate the trajectory: inverse kinematics 
[15] , optimization-based method [17j , and motion capture method |16j . Inverse 
kinematics can generate a trajectory automatically and rapidly; however, this 
method could not generate a collision-free path easily. In order to overcome 
this inconvenience, an optimization-based approach has been proposed in 
to find a collision-free path iteratively. In comparison to inverse kinematics, 
direct kinematics has been used in the optimization based approach and it 
enhances the computation efficiency. However, sometimes, the path can be 
trapped in a local minimum and it cannot get out from it without external 
intervention. Using haptic device or motion capture method, it is convenient to 
achieve natural movement in a virtual environment |18j . However, the motion 
data obtained from the motion capture has to be processed using motion 
retargeting method to adapt it to the overall population. 

In our research, we are aiming at creating a trajectory planning and eval- 
uation method to improve the product design during the conceptual design 
stage. Virtual human modeling is taken to represent the overall population 
with different anthropometrical data. Ergonomic criterions such as fatigue is 
considered in the multi-objective evaluation. A single objective optimization 
based method is used to generate the trajectory at first. Then, motion capture 
methods or other intervention methods are used to help the algorithm to move 
out from the local minimum. At last, multi-objective evaluation methods are 
going to be used to evaluate the generated trajectory. 
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SOO based algorithm 1 
Get current posture q 

Initialize the trajectory Traj by adding q to tail 

Current position of the end effector P_encl=clirect_kmemalic(q) 

Distance D=Disl(P_end, P_des) 

while D>/.i 

1 . Generate a set of new postures in the neighbourhood (q±hq) 
1 .Select the collision-free postures 
Calculate the distances Dist_set 

3. Find out the nearest collision-free posture q = min(Dist_set) 

4. Update current posture q 

5. Update the distance D 

6. Add the current new posture q to trajectory Traj 

end while 



Fig. 1 Single objective optimization based trajectory planning method 

2 Trajectory planning algorithm 

A virtual human is modeled using the modified Denavit-Hartenberg method 
[TO] with 28 degrees of mobilities to describe the mobility of all the key joints 
around human body. The kinematic information could be described by a set 
of general coordinates, where q is the set of the rotational angles representing 
the positions of each joint. 

2.1 - Single objective optimization 
2.1.1 Trajectory planning algorithm 

This single objective optimization (SOO) based method was proposed by [TT] 
in order to generate the trajectory automatically, and its principle is shown 
in Fig. [T] In this algorithm, the distance between the end effector and the 
destination is chosen as an objective function. For a virtual human, the position 
of the destination is known and the current posture q is also known. A change 
(±<5g) to the current posture configuration is added to obtain several posture 
candidates for the next movement. Candidates without collision to the virtual 
environment are selected out via collision test. The one among the remaining 
candidates with the smallest distance is selected to update the current posture. 



2.1.2 Limitations of the algorithm 

One of the greatest technical problems in this method is the local minimum 
encountered while searching the direction to the destination. This problem is 
illustrated by a simple example in Fig. [5J During the trajectory planning, it is 
very possible that the optimization process will encounter the local minimum. 
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Consequently, this algorithm will be trapped and cannot advance anymore to 
its global minimum. In this case, the step length Sq can be modified to skip 
the local minimum, or the configuration q can be changed by another posture 
configuration. These modifications need to be done using external intervention 
[13) . In the situation of Scara Robot, its working space is in 2-dimension space. 
This problem is illustrated by a simple example in FigJ5] In the situation of 
Staubli Robot, its working space is in 3-dimension space. The problem could 
be illustrated in FigJ3] During the trajectory planning, it is very possible that 
the optimization process will encounter the local minimum. In this case, this 
algorithm will be trapped and cannot advance anymore to its global minimum. 
In this case, the step length Sq can be modified to skip the local minimum, or 
the configuration q can be changed by another posture configuration. These 
modifications need to be done using external intervention. 



2.2 Trajectory planning via external intervention 
2.2.1 Modified algorithm 

As what has been discussed in the section 12.11 the single objective optimiza- 
tion method cannot avoid local minimum and that results in no evolution for 
finding a trajectory to the destination. Therefore, a modified algorithm is pro- 
posed in this section using external intervention to overcome this difficulty. 
The algorithm is presented in Fig. @] Since the step length is constant without 
intervention, if a posture q has appeared again in the trajectory, there comes 
local minimum in the trajectory. 
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Fig. 3 Local minimum encountered in 3-dimension space in single objective optimization 
based method 



Modified SOO based Algorithm 
Get current posture q 

Initialize the trajectory Traj by adding q to tail 

Current position of the end effector P_end=direct_kinematic(q) 

Distance D=Dist(P_end, P_des) 

while D>/j 

1 . Generate a set of new postures in the neighbourhood (q±8q) 
2 .Select the collision-free postures 
Calculate the distances Dist_set 

3. Find out the nearest collision-free posture q = min(Dist_set) 
if q exists in the trajectory (local minimum) and D>/i 

Get a new q from external intervention 

end if 

4. Update current posture q 

5. Update the distance D 

6. Add the current new posture q to the trajectory Traj 

end while 



Fig. 4 Modified SOO based method 
2.2.2 Limitations of the algorithm 

An external intervention is implemented via different methods. In this part, a 
multi-agent thought is introduced into our system. The thought is explained 
by Fig. m 

In a virtual space, just using algorithm along could plan a path for the vir- 
tual human (Fig.(SJa)). At the same time, operating the virtual human directly 
by changing its rotational configuration manually could also avoid the obsta- 
cles to get to the target position (Fig. [SJb)). Each way has its disadvantages: 
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(a) algorithm alone (b) operator alone 




(c) co-operation 



Fig. 5 Co-operation principles in multi-agent trajectory planning 



using Fig. [Sfa) will encounter the local minimal obviously; using Fig. EJb) is 
difficult to generate human-like action and much more time consuming. For 
these reasons, the co-operator principle (Fig. [3c)) is implemented by us to 
combine the advantages of each way to get a better result [I] . In order to solve 
these problems, the direct method is to provide a graphical interface to change 
the configuration: the posture q or the step length Sq. Through changing these 
parameters the mannequin can be lead to go out of the local minimum sit- 
uation. Another solution is using motion capture method to skip the local 
minimum. 

In the first solution, changing the posture q can guide the mannequin out 
of the local minimum situation, but it should be known that it is difficult to 
have an intuitive manipulation to operate the virtual human by changing the 
rotational configuration. It requires much more time to manipulate the angular 
variables directly. Enlarging the step length Sq can also guide the mannequin 
to move out the local minimum situation, but the path will not be so smooth. 

In the second solution from a motion tracking system, different motion 
data of a human body can be obtained. The virtual human could be operated 
much more naturally to skip the local minimum. In the second method, a 
motion tracking system is required. Accompanying with this method, inverse 
kinematics or motion retargeting techniques have to be developed to map the 
motion data to the simulated trajectory. 

The main problem is now to define the algorithm which is able to use 
the principle of the multi-agent system [T] and to add the information of the 
motion capture system. 
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2.3 Multi-objective evaluation 



2.3.1 Ergonomic objectives for evaluating the constrained environment 

As we have mentioned in section [TJ there are different aspects that the de- 
signers have to respect. To produce a well designed constrained environment, 
visibility and accessibility are both important factors. Besides them, the phys- 
ical influence from the environment should also be assessed in some cases. 
Therefore, a multi-criteria evaluation system is proposed in this section to 
evaluate the constrained environment. 

Accessibility: this term describes whether the user could obtain an access to 
a certain component in the environment. It could be evaluated by the number 
of possible trajectory solutions (N). The larger the number of solutions, the 
easier the component can be accessed. If there is no solution for the trajectory, 
the component is not accessible by a human being. 

Visibility: this term describes the visual accessibility of a component. This 
term has been modeled or used to analyze workspace in the literature [1] 
[7HH]. In our research, the visibility is going to be integrated into trajectory 
planning by treating it as one of the end effectors, since the visible region is 
also an important factor determining the feasibility of the operation. 

Posture effect: this term describes the effect resulting from posture. This is 
a traditional subject in ergonomic analysis, and there are several conventional 
methods for evaluating the posture [6]. During the ergonomic application, 
duration of the task, posture engaged in the task, and its physical exposures 
are taken to evaluate the potential risks of the posture. 

Fatigue: this term is used to describe the effect of physical load on the hu- 
man body. It has been modeled in [5] according to physical exposures related to 
the manual handling operations. This term is used in our research to evaluate 
the physical effects of the task realized in the constrained environment. 



2.3.2 Fatigue evaluation 



A muscle model and a recovery model were introduced in [5]. These models 
characterize the capability of a human joint thanks to the knowledge of its 
history. The fatigue model is defined the following differential Equation 

dF cem (t) F cem (t) 

~^r~ = ~ ~mvc~ ° a (1) 

at 

Related parameters and their descriptions are given in Table [1] These models 
describe the muscle fatigue mechanism from a macro aspect based on the 
muscle motor unit recruitment principle. 

The capacity of each human joint can be described by Eqns.[T]and[21 For 
the evaluation of trajectory, a new index is introduce to aggregate the status 
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Table 1 Parameters in muscle model and a recovery model 



Item 


Unit 


Description 


MVC 


N 


Maximum voluntary contraction, maximum capacity of muscle, 


Fcem (^) 


N 


Current exertable maximum force, current capacity of muscle 


FloadiS) 


N 


External load of muscle, the force which the muscle needs to 






generate 


k 


min~ 


Constant value, fatigue ratio, here k = 1 



%MVC Percentage of the voluntary maximum contraction 

fuvc %MVC /WO, f MVC = ^L. 

R Recovery rate from fatigue model motor units, R = 2.4 1141 



of each human joint relative to their endurance limit. The remaining forces 
are defined as, in percentage 

. ( F cem (t) — Fi j0a d i \ , . 

Fr = mm — ; .100 (3) 

where F cemi , F^oa^ and Fuvd defined the capacity associated with the hu- 
man joint i. These capacities can be divided into two subgroups for the same 
joint to define the push/pull capacity. 

2.3.3 Technical approach 

As discussed before, multi-criteria evaluation system is going to be established 
to assess different ergonomic aspects of a constrained environment. In this 
approach, different aspects are mathematically modeled to create objective 
evaluation. Those results could be useful to improve the design of constrained 
environment. 

Meanwhile, a multi-objective optimization procedure could also be inter- 
esting to determine design parameters of a constrained environment. This 
algorithm is presented in Fig. [5J 

3 Case study 

3.1 Hardware and software implementation 

In order to realize our algorithm, a virtual reality platform is constructed. 
This platform includes two parts: simulation system and motion capture sys- 
tem. The simulation system is mainly responsible for the generation of virtual 
environment, the collision computation, and the automatic trajectory plan- 
ning. An optical motion tracking system is in charge of capturing motion data 
and communicating with the simulation system. 

The simulation system is developed using OpenGL and C++. The virtual 
skeleton is shown graphically in Fig. [7] The virtual human is combined by ten 
body parts: head, torso, thighs, shanks, upper arms, and lower arms. Each 
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MOO based Algorithm 


1. 


A design D need to be evaluated 


2. 


A set of trajectories T geneiated according to different 




aspects 


3. 


Different aspects of the trajectory 




a. Visibility evaluation 




b. Accessibility evaluation 




c . Posture evaluation 




d. Physical evaluation 


4. 


Analysis of the evaluation results 




a. Evaluate current design 




b. Find out possible optimization directions 


5. 


Update of the design D if necessary 



Fig. 6 Multi-objective optimization (MOO) based evaluation and trajectory planning al- 
gorithm 




Fig. 7 The virtual skeleton in the simulation system 



body part is modeled as a 3DS model file which is composed of hundreds of 
triangle facts. The virtual skeleton is driven using direct kinematic method by 
changing angular values of each key joints. Meanwhile, virtual environments 
could also be loaded from 3DS files which are converted directly from CAD 
models. 

In the motion capture system (Fig. [8]), there are totally eight CCD cam- 
eras to capture the motion in a range of 2m x 2m x 2m. Nonlinear direct 
transformation method is used to calibrate all the cameras. After calibration, 
the system could capture maximum 13 markers at 25Hz 8 . Although this 
tracking speed is not enough for capturing accurate motion, it still provides 
an acceptable speed to adjust the posture. 
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Fig. 8 Optical motion capture system 



3.2 Robot trajectory planning for Scara Robot 

Since there are too many degrees of mobilities in a virtual human, at the 
very beginning of our research, a trial demonstration of the algorithm has 
been realized by using a Scara robot and several virtual objects in a virtual 
environment (see Fig. IMTT1). The robot is composed of three rectangles, and 
the end effector is the right end. The blue round point denotes the destination 
of the end effector. There are three obstacles (two triangles and one block) in 
the environment. 

Different trajectories will be generated by using our algorithm according 
to the different length of each revolute joint in the Scara robot. Figure [9] 
shows that trajectory of Scara robot with the link length parameters (20, 10, 
20) . Figure [TU] shows that the trajectory of Scara robot with the link length 
parameters (20, 20, 25) and Fig 10 shows that the trajectory of Scara robot 
with the link length parameters (20, 20, 40). From these figures, we can see 
that in the same environment, different size of the Scara robot can come across 
various situations. This is necessary and also important, because of a product 
is not just for a fixed user. Various situation or parameters of a subject should 
be taken into consideration. The lengths of the links are changed demonstrate 
different effects of dimensional information. 

The trajectory in blue is generated by the algorithm presented in section 
12.1.11 It is observable that the trajectory in blue could avoid the collision 
while approaching to the destination. Figures M ITUl and [Til show that with the 
same obstacle different geometrical configurations (different arm lengths) can 
generate different paths to avoid the obstacle from the same start to the same 
destination. 
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Fig. 9 Trajectory planning test using a Scara robot (with link length equals 20, 10, 20) 
Fig. 10 Trajectory planning test using a Scara robot (with link length equals 20, 20, 25) 




Fig. 11 Trajectory planning test using a Scara robot (with link length equals 20, 20, 40) 

In the obstacle avoiding process, there is always possibility of local mini- 
mum. In Fig. 1121 a demonstration of local minimum is shown. 

Since the obstacles locate between the destination and the end effector and 
the descending direction is also restricted by the obstacles, the algorithm could 
not skip the local minimum with a step length 0.08. 

In Fig. [13l for the same arrangement of obstacles in Fig 10, the step length 
has been adjusted to 0.1. As a result, the first obstacle could be passed over 
without problem. 



3.3 Robot trajectory planning for Staubli Robot 

In the trial demonstration of the Scara Robot, its working space is in 2- 
dimensional. For each articulation it just has one degree of freedom. In this 
situation, the local minimum problem is usually happened. The second trial of 
the algorithm is in the Staubli robot, we use the part of shoulder and elbow, 
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Fig. 12 Trajectory planning test us- Fig. 13 Trajectory planning test using a 
ing a Scara robot (with local minimum, Scara robot (External intervention inter- 
step=0.08) face, step=0.1) 




Fig. 14 Geometry model of arm 

Table 2 Geometry modeling parameters of arm 



3 


°j 


a 3 


d 3 


&i 


r j 


1 











0i 





2 





tt/2 





02 





3 








d 3 


6*3 





4 








d 4 









in shoulder it has 2 degrees of freedom and in the elbow it has 1 degree of 
freedom. We do this trial demonstration because of this robot is more similar 
to the human arm. The Arm geometric model is in Fig. [TJJ The geometry 
modelling parameter of arm is in Fig. 1141 dj, is the length of upper arm, d± is 
the length of forearm. B\ , 62 is the rotation angle of shoulder in horizontal and 
vertical direction. 83 is the rotation angle of elbow. 
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Fig. 15 Trajectory planning test using a Staubli robot (with link length equals 40,30) 



V 




Fig. 16 Local minimum problem of Staubli robot 

Based on the Staubli robot, 3-dimensional trajectorie will be generated by 
using our algorithm. Figure 1151 shows that trajectory of Staubli robot with 
the link length parameters (40, 30) when it want to reach the destination how 
to avoid a spatial obstacle and the trajectory is in 3-dimenstional. The black 
line is the trajectory. Because of the Staubli robot in the shoulder has 2 degree 
of freedom, when it meet the obstacle, it has one more dimension direction to 
choose, this means that it has more chance to avoid the obstacle compared 
with the Scara Robot. Although the 3-dimensional direction could improve 
the solution when the arm meets the obstacle, it also has the possibility to be 
locked in the local minimum problem. The Fig. [131 shows that the obstacle is 
composed by 2 triangles, and it forms a concave shape. When the arm moves 
inside the concave, it is locked. From the Fig. [TBI we could see that it repeat the 
irregular trajectory inside the concave. After change the step=0.06, it jumps 
out the local minimum range and move to the destination. 



3.4 Virtual human trajectory planning 

Virtual human trajectory planning using the proposed algorithm is still under 
construction. There are still several steps to complete the demonstration: mo- 
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1 



a 




Fig. 17 A set of mannequins with different percentiles to test new products 




Fig. 18 Example of office design for two different female cases 



tion retargeting, modeling of different aspects, and the complete installation 
of the motion capture system. 

The definition of the trajectories should be not just used for one person, 
because of a product is designed for a given population (Fig.[T7]). An automatic 
path planner can calculate the path from a start point to a destination. But 
imaging that in a complex environment and for many users, sometimes the 
algorithm might be failed. In this situation, the user interaction has to be 
limited to minimize design effort. 

The ergonomic study of the product can be tested thanks to the defini- 
tion of a set of tasks: taking a mouse, touching the screen (Fig. [18]). Different 
tasks have different aspects to be respected, and different aspects will result 
different trajectories. Different sizes, different weights, different tasks, and dif- 
ferent design aspects: all these factors lead trajectory planning for a virtual 
human to a quite difficult problem. The multi-objective evaluation and opti- 
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mization approach for virtual human trajectory planning has to be developed 
with caution. 



4 Conclusions and perspectives 

In this paper, an approach of using virtual human in constrained environment 
design was presented, and a new algorithm was proposed to solve trajectory 
planning for two types of robot. Our preliminary application of the algorithm 
demonstrates that this interactive method had a potential to help us solve 
trajectory planning problem in constrained environment for mannequins. In 
our future research, a multi-agent system and multi-objective optimization 
method will be implemented to facilitate the design process of constrained 
environments. 
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